Video abstract for "An Adaptive Homeostatic Algorithm for the Unsupervised Learning of Visual Features"

This video shows the results of unsupervised learning with different type of kezrnel normalization. This is to illustrate the results obtained in this paper on the An Adaptive Homeostatic Algorithm for the Unsupervised Learning of Visual Features which is now in press.



In [1]:
%matplotlib inline
In [8]:
%ls -ltr /tmp/cache_dir
total 966808
-rw-r--r--  1 laurentperrinet  wheel  231154688 Sep 11 18:15 2019-09-11_Perrinet19_data_data.npy
-rw-r--r--  1 laurentperrinet  wheel    5496750 Sep 11 18:16 2019-09-11_Perrinet19_None_0_dico.pkl
-rw-r--r--  1 laurentperrinet  wheel    5496793 Sep 11 18:17 2019-09-11_Perrinet19_None_1_dico.pkl
-rw-r--r--  1 laurentperrinet  wheel    5496793 Sep 11 18:17 2019-09-11_Perrinet19_None_2_dico.pkl
-rw-r--r--  1 laurentperrinet  wheel    5496793 Sep 11 18:18 2019-09-11_Perrinet19_None_3_dico.pkl
-rw-r--r--  1 laurentperrinet  wheel    5496793 Sep 11 18:19 2019-09-11_Perrinet19_None_4_dico.pkl
-rw-r--r--  1 laurentperrinet  wheel    5496793 Sep 11 18:20 2019-09-11_Perrinet19_None_5_dico.pkl
-rw-r--r--  1 laurentperrinet  wheel    5496793 Sep 11 18:20 2019-09-11_Perrinet19_None_6_dico.pkl
-rw-r--r--  1 laurentperrinet  wheel    5496793 Sep 11 18:21 2019-09-11_Perrinet19_None_7_dico.pkl
-rw-r--r--  1 laurentperrinet  wheel    5496793 Sep 11 18:22 2019-09-11_Perrinet19_None_8_dico.pkl
-rw-r--r--  1 laurentperrinet  wheel    5496793 Sep 11 18:23 2019-09-11_Perrinet19_None_9_dico.pkl
-rw-r--r--  1 laurentperrinet  wheel    5496793 Sep 11 18:24 2019-09-11_Perrinet19_None_10_dico.pkl
-rw-r--r--  1 laurentperrinet  wheel    5496793 Sep 11 18:24 2019-09-11_Perrinet19_None_11_dico.pkl
-rw-r--r--  1 laurentperrinet  wheel    5496750 Sep 11 18:25 2019-09-11_Perrinet19_OLS_0_dico.pkl
-rw-r--r--  1 laurentperrinet  wheel    5496793 Sep 11 18:26 2019-09-11_Perrinet19_OLS_1_dico.pkl
-rw-r--r--  1 laurentperrinet  wheel    5496793 Sep 11 18:27 2019-09-11_Perrinet19_OLS_2_dico.pkl
-rw-r--r--  1 laurentperrinet  wheel    5496793 Sep 11 18:27 2019-09-11_Perrinet19_OLS_3_dico.pkl
-rw-r--r--  1 laurentperrinet  wheel    5496793 Sep 11 18:28 2019-09-11_Perrinet19_OLS_4_dico.pkl
-rw-r--r--  1 laurentperrinet  wheel    5496793 Sep 11 18:29 2019-09-11_Perrinet19_OLS_5_dico.pkl
-rw-r--r--  1 laurentperrinet  wheel    5496793 Sep 11 18:30 2019-09-11_Perrinet19_OLS_6_dico.pkl
-rw-r--r--  1 laurentperrinet  wheel    5496793 Sep 11 18:31 2019-09-11_Perrinet19_OLS_7_dico.pkl
-rw-r--r--  1 laurentperrinet  wheel    5496793 Sep 11 18:31 2019-09-11_Perrinet19_OLS_8_dico.pkl
-rw-r--r--  1 laurentperrinet  wheel    5496793 Sep 11 18:32 2019-09-11_Perrinet19_OLS_9_dico.pkl
-rw-r--r--  1 laurentperrinet  wheel    5496793 Sep 11 18:33 2019-09-11_Perrinet19_OLS_10_dico.pkl
-rw-r--r--  1 laurentperrinet  wheel    5496793 Sep 11 18:34 2019-09-11_Perrinet19_OLS_11_dico.pkl
-rw-r--r--  1 laurentperrinet  wheel    5496750 Sep 11 18:34 2019-09-11_Perrinet19_HEH_0_dico.pkl
-rw-r--r--  1 laurentperrinet  wheel    5496793 Sep 11 18:35 2019-09-11_Perrinet19_HEH_1_dico.pkl
-rw-r--r--  1 laurentperrinet  wheel    5496793 Sep 11 18:36 2019-09-11_Perrinet19_HEH_2_dico.pkl
-rw-r--r--  1 laurentperrinet  wheel    5496793 Sep 11 18:37 2019-09-11_Perrinet19_HEH_3_dico.pkl
-rw-r--r--  1 laurentperrinet  wheel    5496793 Sep 11 18:37 2019-09-11_Perrinet19_HEH_4_dico.pkl
-rw-r--r--  1 laurentperrinet  wheel    5496793 Sep 11 18:38 2019-09-11_Perrinet19_HEH_5_dico.pkl
-rw-r--r--  1 laurentperrinet  wheel    5496793 Sep 11 18:39 2019-09-11_Perrinet19_HEH_6_dico.pkl
-rw-r--r--  1 laurentperrinet  wheel    5496793 Sep 11 18:40 2019-09-11_Perrinet19_HEH_7_dico.pkl
-rw-r--r--  1 laurentperrinet  wheel    5496793 Sep 11 18:41 2019-09-11_Perrinet19_HEH_8_dico.pkl
-rw-r--r--  1 laurentperrinet  wheel    5496793 Sep 11 18:41 2019-09-11_Perrinet19_HEH_9_dico.pkl
-rw-r--r--  1 laurentperrinet  wheel    5496793 Sep 11 18:42 2019-09-11_Perrinet19_HEH_10_dico.pkl
-rw-r--r--  1 laurentperrinet  wheel    5496793 Sep 11 18:43 2019-09-11_Perrinet19_HEH_11_dico.pkl
-rw-r--r--  1 laurentperrinet  wheel    5496750 Sep 11 18:44 2019-09-11_Perrinet19_HAP_0_dico.pkl
-rw-r--r--  1 laurentperrinet  wheel    5496793 Sep 11 18:44 2019-09-11_Perrinet19_HAP_1_dico.pkl
-rw-r--r--  1 laurentperrinet  wheel    5496793 Sep 11 18:45 2019-09-11_Perrinet19_HAP_2_dico.pkl
-rw-r--r--  1 laurentperrinet  wheel    5496793 Sep 11 18:46 2019-09-11_Perrinet19_HAP_3_dico.pkl
-rw-r--r--  1 laurentperrinet  wheel    5496793 Sep 11 18:47 2019-09-11_Perrinet19_HAP_4_dico.pkl
-rw-r--r--  1 laurentperrinet  wheel    5496793 Sep 11 18:47 2019-09-11_Perrinet19_HAP_5_dico.pkl
-rw-r--r--  1 laurentperrinet  wheel    5496793 Sep 11 18:48 2019-09-11_Perrinet19_HAP_6_dico.pkl
-rw-r--r--  1 laurentperrinet  wheel    5496793 Sep 11 18:49 2019-09-11_Perrinet19_HAP_7_dico.pkl
-rw-r--r--  1 laurentperrinet  wheel    5496793 Sep 11 18:50 2019-09-11_Perrinet19_HAP_8_dico.pkl
-rw-r--r--  1 laurentperrinet  wheel    5496793 Sep 11 18:50 2019-09-11_Perrinet19_HAP_9_dico.pkl
-rw-r--r--  1 laurentperrinet  wheel    5496793 Sep 11 18:51 2019-09-11_Perrinet19_HAP_10_dico.pkl
-rw-r--r--  1 laurentperrinet  wheel    5496793 Sep 11 18:52 2019-09-11_Perrinet19_HAP_11_dico.pkl
In [9]:
%ls -ltr /tmp/2019-09-11*
ls: /tmp/2019-09-11*: No such file or directory
In [4]:
%rm /tmp/cache_dir/*lock* 
#%rm/tmp/2019-09-11*
rm: /tmp/cache_dir/*lock*: No such file or directory
In [94]:
d_new, d_old = 1, 1
N_step = 16
for i_step in range(N_step):
    print(d_new, d_old)
    d_new, d_old = d_new + d_old, d_new
1 1
2 1
3 2
5 3
8 5
13 8
21 13
34 21
55 34
89 55
144 89
233 144
377 233
610 377
987 610
1597 987
In [16]:
%%writefile /tmp/video_abstract.py
#initialize
import os
import numpy as np
np.set_printoptions(precision=2, suppress=True)
seed = 42
np.random.seed(seed)

from shl_scripts.shl_experiments import SHL
#from shl_scripts import sparse_encode
name = '2019-09-11_Perrinet19'

datapath = '/tmp/database'
opts = dict(datapath=datapath, verbose=0, cache_dir='/tmp/cache_dir', n_iter=2**5 + 1, seed=seed)
shl = SHL(**opts)
data = shl.get_data(matname=f'{name}_data')

# running main simulations
# Figure 1 & 3
N_cv = 1
homeo_methods = ['None', 'OLS', 'HEH', 'HAP']

n_jobs = 1
N_step = 12
N_step = 16
    
from shl_scripts import touch
#from shl_scripts.shl_experiments import SHL_set
for homeo_method, color in zip(homeo_methods, colors):
    print(f'=> homeo_method={homeo_method}')
    flockname = os.path.join(shl.cache_dir, f'{name}_{homeo_method}_lock')
    if not os.path.isfile(flockname):
        touch(fmatname + '_lock')
        # TODO use fibonacci
        d_new, d_old = 1, 1
        
        for i_step in range(N_step):

            opts_ = opts.copy()
            n_iter = int(base_iter**i_step + 1)

            print(f'==> i_iter={i_step} / n_iter={d_new}')
            opts_.update(homeo_method=homeo_method, n_iter=d_old,)

            shl = SHL(**opts)
            if i_step==0:
                dictionary = None
            else:
                dictionary = shl.learn_dico(data=data, matname=f'{name}_{homeo_method}_{i_step-1}', list_figures=[]).dictionary

            dico = shl.learn_dico(data=data, dictionary=dictionary, matname=f'{name}_{homeo_method}_{i_step}', list_figures=[])

            d_new, d_old = d_new + d_old, d_new
                          
        if os.path.isfile(flockname): os.remove(flockname)

                          
Overwriting /tmp/video_abstract.py
In [15]:
%run  /tmp/video_abstract.py
  File "/tmp/video_abstract.py", line 32
    if not os.path.isfile(flockname:
                                   ^
SyntaxError: invalid syntax
<Figure size 432x288 with 0 Axes>
In [69]:
homeo_methods
Out[69]:
['None', 'OLS', 'HEH', 'HAP']
In [72]:
phi = (np.sqrt(5) + 1. ) /2
fig_width = 15
dpi_export = 300
#dpi_export = 0
colors = ['black', 'orange', 'blue', 'red']
In [ ]:
import matplotlib.pyplot as plt
def plot_dico(dico, color, dim_graph = (8, 8)):
    subplotpars = dict(left=0.05, right=.95, bottom=0.05, top=.95, wspace=0.05, hspace=0.05,)
    fig, axs = plt.subplots(1, 2, figsize=(fig_width, fig_width/(1+phi)), gridspec_kw=subplotpars)
    for ax in axs:
        ax.axis(c='b', lw=2, axisbg='w')
        ax.set_facecolor('w')
        
    from shl_scripts import show_dico
    fig, axs[0] = show_dico(shl, dico, data=data, dim_graph=dim_graph, fig=fig, ax=axs[0], order=False)
    
    if False:
        from shl_scripts import plot_P_cum
        fig, axs[1] = plot_P_cum(dico.P_cum, ymin=0.6, ymax=1.001, 
                             title=None, suptitle=None, ylabel='non-linear functions', 
                             verbose=False, n_yticks=21, alpha=.02, c=color, fig=fig, ax=axs[1])
        axs[1].plot([0], [0], lw=1, color=color, label=homeo_method, alpha=.6)
    
    else:
        from shl_scripts import plot_proba_histogram
        coding = shl.code(data, dico)
        fig, axs[1] = plot_proba_histogram(coding, fig=fig, ax=axs[1])
        axs[1].plot([0], [0], lw=1, color=color, label=homeo_method, alpha=.6)
    # ax.set_ylabel(homeo_method)

    #ax.text(-8, 7*dim_graph[0], homeo_method, fontsize=12, color='k', rotation=90)#, backgroundcolor='white'
    #axs[1].legend(loc='lower right')
    return fig, axs

for homeo_method, color in zip(homeo_methods, colors):
    print(f'=> homeo_method={homeo_method}')

    for i_step in range(N_step):
        print(f'==> i_iter={i_step} / n_iter={n_iter}')
        figname = f'/tmp/{name}_{homeo_method}_{i_step}.png'
        if not os.path.isfile(figname):
            fig, axs = plot_dico(dico, color)
            axs[1].set_ylim(0. 1.7)
            axs[1].text(10, 0.1, f'homeo_method={homeo_method}', fontsize=24, color=color, rotation=0)#, backgroundcolor='white'
            axs[1].text(10, 1.5, f'n_iter={n_iter}', fontsize=24, color='k', rotation=0)#, backgroundcolor='white'
            plt.show()
            if dpi_export > 0 : fig.savefig(figname, dpi=dpi_export, bbox_inches='tight')

                          
=> homeo_method=None
==> i_iter=0 / n_iter=2
==> i_iter=1 / n_iter=2
==> i_iter=2 / n_iter=3
==> i_iter=3 / n_iter=5
==> i_iter=4 / n_iter=7
==> i_iter=5 / n_iter=12
==> i_iter=6 / n_iter=18
==> i_iter=7 / n_iter=30
==> i_iter=8 / n_iter=47
==> i_iter=9 / n_iter=77
==> i_iter=10 / n_iter=123
==> i_iter=11 / n_iter=200
=> homeo_method=OLS
==> i_iter=0 / n_iter=2
==> i_iter=1 / n_iter=2
==> i_iter=2 / n_iter=3
==> i_iter=3 / n_iter=5
==> i_iter=4 / n_iter=7
==> i_iter=5 / n_iter=12
==> i_iter=6 / n_iter=18
==> i_iter=7 / n_iter=30
==> i_iter=8 / n_iter=47
==> i_iter=9 / n_iter=77
==> i_iter=10 / n_iter=123
==> i_iter=11 / n_iter=200
=> homeo_method=HEH
==> i_iter=0 / n_iter=2
==> i_iter=1 / n_iter=2
==> i_iter=2 / n_iter=3
==> i_iter=3 / n_iter=5
==> i_iter=4 / n_iter=7
==> i_iter=5 / n_iter=12
==> i_iter=6 / n_iter=18
==> i_iter=7 / n_iter=30
==> i_iter=8 / n_iter=47
==> i_iter=9 / n_iter=77
==> i_iter=10 / n_iter=123
==> i_iter=11 / n_iter=200
=> homeo_method=HAP
==> i_iter=0 / n_iter=2
==> i_iter=1 / n_iter=2
==> i_iter=2 / n_iter=3
==> i_iter=3 / n_iter=5
==> i_iter=4 / n_iter=7
==> i_iter=5 / n_iter=12
==> i_iter=6 / n_iter=18

Creating the movie using the (excellent) MoviePy library:

In [39]:
[f'/tmp/{name}_{homeo_method}_{i_step}.png' for i_step in range(N_step)]
Out[39]:
['/tmp/2019-09-11_Perrinet19_HAP_0.png',
 '/tmp/2019-09-11_Perrinet19_HAP_1.png',
 '/tmp/2019-09-11_Perrinet19_HAP_2.png',
 '/tmp/2019-09-11_Perrinet19_HAP_3.png',
 '/tmp/2019-09-11_Perrinet19_HAP_4.png',
 '/tmp/2019-09-11_Perrinet19_HAP_5.png',
 '/tmp/2019-09-11_Perrinet19_HAP_6.png',
 '/tmp/2019-09-11_Perrinet19_HAP_7.png',
 '/tmp/2019-09-11_Perrinet19_HAP_8.png',
 '/tmp/2019-09-11_Perrinet19_HAP_9.png',
 '/tmp/2019-09-11_Perrinet19_HAP_10.png',
 '/tmp/2019-09-11_Perrinet19_HAP_11.png']
In [42]:
vext = 'mp4'

from moviepy.editor import ImageSequenceClip

for homeo_method in homeo_methods:
    clip = ImageSequenceClip([f'/tmp/{name}_{homeo_method}_{i_step}.png' for i_step in range(N_step)], fps=2)
    clip.write_videofile(f'/tmp/{name}_{homeo_method}.{vext}')
                                                              
t:   7%|▋         | 2/30 [05:24<1:15:39, 162.12s/it, now=None]
                                                              
t:   7%|▋         | 2/30 [05:24<1:15:39, 162.12s/it, now=None]
t:   7%|▋         | 2/30 [05:12<1:12:57, 156.35s/it, now=None]

t:   0%|          | 0/12 [00:00<?, ?it/s, now=None]

t:  17%|█▋        | 2/12 [00:00<00:00, 11.48it/s, now=None]
Moviepy - Building video /tmp/2019-09-11_Perrinet19_None.mp4.
Moviepy - Writing video /tmp/2019-09-11_Perrinet19_None.mp4


t:  25%|██▌       | 3/12 [00:00<00:01,  6.12it/s, now=None]

t:  33%|███▎      | 4/12 [00:00<00:01,  4.79it/s, now=None]

t:  42%|████▏     | 5/12 [00:01<00:01,  4.24it/s, now=None]

t:  50%|█████     | 6/12 [00:01<00:01,  3.92it/s, now=None]

t:  58%|█████▊    | 7/12 [00:01<00:01,  3.73it/s, now=None]

t:  67%|██████▋   | 8/12 [00:02<00:01,  3.59it/s, now=None]

t:  75%|███████▌  | 9/12 [00:02<00:00,  3.53it/s, now=None]

t:  83%|████████▎ | 10/12 [00:02<00:00,  3.49it/s, now=None]

t:  92%|█████████▏| 11/12 [00:02<00:00,  3.44it/s, now=None]

t: 100%|██████████| 12/12 [00:03<00:00,  3.36it/s, now=None]

                                                              A
t:   7%|▋         | 2/30 [05:28<1:16:45, 164.49s/it, now=None]
                                                              
t:   7%|▋         | 2/30 [05:28<1:16:45, 164.49s/it, now=None]
t:   7%|▋         | 2/30 [05:17<1:14:04, 158.71s/it, now=None]
Moviepy - Done !
Moviepy - video ready /tmp/2019-09-11_Perrinet19_None.mp4
                                                              
t:   7%|▋         | 2/30 [05:31<1:17:20, 165.73s/it, now=None]
                                                              
t:   7%|▋         | 2/30 [05:31<1:17:20, 165.73s/it, now=None]
t:   7%|▋         | 2/30 [05:19<1:14:38, 159.96s/it, now=None]

t:   0%|          | 0/12 [00:00<?, ?it/s, now=None]

t:  17%|█▋        | 2/12 [00:00<00:00, 12.31it/s, now=None]
Moviepy - Building video /tmp/2019-09-11_Perrinet19_OLS.mp4.
Moviepy - Writing video /tmp/2019-09-11_Perrinet19_OLS.mp4


t:  25%|██▌       | 3/12 [00:00<00:01,  6.31it/s, now=None]

t:  33%|███▎      | 4/12 [00:00<00:01,  4.92it/s, now=None]

t:  42%|████▏     | 5/12 [00:01<00:01,  4.33it/s, now=None]

t:  50%|█████     | 6/12 [00:01<00:01,  3.97it/s, now=None]

t:  58%|█████▊    | 7/12 [00:01<00:01,  3.77it/s, now=None]

t:  67%|██████▋   | 8/12 [00:01<00:01,  3.65it/s, now=None]

t:  75%|███████▌  | 9/12 [00:02<00:00,  3.55it/s, now=None]

t:  83%|████████▎ | 10/12 [00:02<00:00,  3.49it/s, now=None]

t:  92%|█████████▏| 11/12 [00:02<00:00,  3.44it/s, now=None]

t: 100%|██████████| 12/12 [00:03<00:00,  3.43it/s, now=None]

                                                              A
t:   7%|▋         | 2/30 [05:36<1:18:25, 168.05s/it, now=None]
                                                              
t:   7%|▋         | 2/30 [05:36<1:18:25, 168.06s/it, now=None]
t:   7%|▋         | 2/30 [05:24<1:15:43, 162.28s/it, now=None]
Moviepy - Done !
Moviepy - video ready /tmp/2019-09-11_Perrinet19_OLS.mp4
                                                              
t:   7%|▋         | 2/30 [05:38<1:19:00, 169.31s/it, now=None]
                                                              
t:   7%|▋         | 2/30 [05:38<1:19:00, 169.31s/it, now=None]
t:   7%|▋         | 2/30 [05:27<1:16:18, 163.53s/it, now=None]

t:   0%|          | 0/12 [00:00<?, ?it/s, now=None]

t:  17%|█▋        | 2/12 [00:00<00:00, 12.40it/s, now=None]
Moviepy - Building video /tmp/2019-09-11_Perrinet19_HEH.mp4.
Moviepy - Writing video /tmp/2019-09-11_Perrinet19_HEH.mp4


t:  25%|██▌       | 3/12 [00:00<00:01,  6.45it/s, now=None]

t:  33%|███▎      | 4/12 [00:00<00:01,  4.95it/s, now=None]

t:  42%|████▏     | 5/12 [00:01<00:01,  4.31it/s, now=None]

t:  50%|█████     | 6/12 [00:01<00:01,  3.93it/s, now=None]

t:  58%|█████▊    | 7/12 [00:01<00:01,  3.70it/s, now=None]

t:  67%|██████▋   | 8/12 [00:02<00:01,  3.56it/s, now=None]

t:  75%|███████▌  | 9/12 [00:02<00:00,  3.47it/s, now=None]

t:  83%|████████▎ | 10/12 [00:02<00:00,  3.41it/s, now=None]

t:  92%|█████████▏| 11/12 [00:02<00:00,  3.37it/s, now=None]

t: 100%|██████████| 12/12 [00:03<00:00,  3.33it/s, now=None]

                                                              A
t:   7%|▋         | 2/30 [05:43<1:20:06, 171.67s/it, now=None]
                                                              
t:   7%|▋         | 2/30 [05:43<1:20:06, 171.67s/it, now=None]
t:   7%|▋         | 2/30 [05:31<1:17:25, 165.89s/it, now=None]
Moviepy - Done !
Moviepy - video ready /tmp/2019-09-11_Perrinet19_HEH.mp4
                                                              
t:   7%|▋         | 2/30 [05:45<1:20:42, 172.95s/it, now=None]
                                                              
t:   7%|▋         | 2/30 [05:45<1:20:42, 172.95s/it, now=None]
t:   7%|▋         | 2/30 [05:34<1:18:00, 167.18s/it, now=None]

t:   0%|          | 0/12 [00:00<?, ?it/s, now=None]

t:  17%|█▋        | 2/12 [00:00<00:00, 11.93it/s, now=None]
Moviepy - Building video /tmp/2019-09-11_Perrinet19_HAP.mp4.
Moviepy - Writing video /tmp/2019-09-11_Perrinet19_HAP.mp4


t:  25%|██▌       | 3/12 [00:00<00:01,  6.11it/s, now=None]

t:  33%|███▎      | 4/12 [00:00<00:01,  4.80it/s, now=None]

t:  42%|████▏     | 5/12 [00:01<00:01,  4.20it/s, now=None]

t:  50%|█████     | 6/12 [00:01<00:01,  3.88it/s, now=None]

t:  58%|█████▊    | 7/12 [00:01<00:01,  3.69it/s, now=None]

t:  67%|██████▋   | 8/12 [00:02<00:01,  3.56it/s, now=None]

t:  75%|███████▌  | 9/12 [00:02<00:00,  3.49it/s, now=None]

t:  83%|████████▎ | 10/12 [00:02<00:00,  3.45it/s, now=None]

t:  92%|█████████▏| 11/12 [00:02<00:00,  3.43it/s, now=None]

t: 100%|██████████| 12/12 [00:03<00:00,  3.41it/s, now=None]

                                                              A
t:   7%|▋         | 2/30 [05:50<1:21:48, 175.32s/it, now=None]
                                                              
t:   7%|▋         | 2/30 [05:50<1:21:48, 175.32s/it, now=None]
t:   7%|▋         | 2/30 [05:39<1:19:07, 169.54s/it, now=None]
Moviepy - Done !
Moviepy - video ready /tmp/2019-09-11_Perrinet19_HAP.mp4

Annotating the movie using the (excellent) MoviePy library:

In [81]:
from moviepy.editor import VideoFileClip, ImageClip, TextClip, CompositeVideoClip


H = 500
W = 800
SIZE = (W, H)


#clip = [VideoFileClip(f'/tmp/{name}_{homeo_method}.{vext}')]
clip = []
t = 0 
texts = ["An Adaptive Homeostatic Algorithm\n for the Unsupervised Learning\n of Visual Features",
         "as published Sep 2019\n in the journal Vision"
        ]
colors_intro = ['orange', 'white']

txt_opts = dict(fontsize=50, font="Amiri-Bold", stroke_color='gray', stroke_width=.5, size=(W,H))
duration = 3
for text, color in zip(texts, colors_intro):
    txt_clip = TextClip(text, color=color, **txt_opts)
    txt_clip = txt_clip.set_start(t).set_pos('center').set_duration(duration)#.resize(height=H, width=W)
    
    t += duration
    clip.append(txt_clip)
    
In [74]:
colors
Out[74]:
['black', 'orange', 'blue', 'red']
In [82]:
texts = ["No homeostasis", "Olshausen (1997)", "Histogram Equalization", "Activation Probability"]
subtitles = {}
subtitles['None'] = [
    'I show here the result of learning...',
    '... evolving with the number of iterations  ...',
    '... on the left the receptive fields (kernels) ...',
    '... by showing 64 randomly picked from the 661 cells ...',
    '... and on the right histogram of activation probability.',
    '... After some iterations, edge-like filters appear ...',
    '... but the histogram shows a desequilibrium in the ...',
    '... activation probability, even though  ...',
    '... the norm of the kerenals are normalized.',
]
subtitles['OLS'] = [
    'The original homeostasis algorithm developped ...',
    '... by Olshausen and Field (1997) included a more  ...',
    '... elaborated adaptation based on the variance of ...',
    '... the coefficients, yielding a better balance...',
    '...  ...',
    'Still, this causes an under-optimisation of the ... ',
    '... sparse code as some are a priori ...',
    '... more likely to be selected than ...',
    '... others ...',
]
subtitles['HEH'] = [
    'Such an objective can be simply optimzed...',
    '... by introducing an adaptive non-linearity in our ...',
    '... sparse coding algorithm. This non-linearity  ...',
    '... is based on the concept of histogram normalization ...',
    '... to make sure that all neurons are always picked up.',
    '...  with a priori the same probability ...',
    '... We called this Histogram Equalization Homeostasis ...',
    '... and it effectively works very well, as well ...',
    '... qualitatively (left) than quantitatively (right).',
]
subtitles['HAP'] = [
    'A problem of Histogram Equalization Homeostasis ...',
    '... is that it is computationally more expensive  ...',
    '... and less plausible to be implemented in real ...',
    '...  biological neurons. We thus explored another...',
    '... algorithm simply based on the activation probability.',
    'This Homeostasis on the Activation Probability  ... ',
    '... performed qualitatively as well as HEH ...',
    '... and also quantitatively similarly ...',
    'Yet it comes at a very low cost compatible with biomimetic algorithms',
]

txt_opts = dict(fontsize=20, font="Amiri-Bold", stroke_color='gray', stroke_width=.5, size=(W,H))
for homeo_method, text, color in zip(homeo_methods, texts, colors):
    print(f'=> homeo_method={homeo_method}')
    duration = 1
    txt_clip = TextClip(text, color=color, **txt_opts)
    txt_clip = txt_clip.set_start(t).set_pos('center').set_duration(duration)
    t += duration
    clip.append(txt_clip)
    
    #clip = [VideoFileClip(f'{name}_{homeo_method}.{vext}')]
    
    for i_step in range(N_step):
        duration = 1
        im_clip = ImageClip(f'/tmp/{name}_{homeo_method}_{i_step}.png')
        im_clip = im_clip.set_start(t).set_duration(duration)
        t += duration
        clip.append(im_clip)
=> homeo_method=None
=> homeo_method=OLS
=> homeo_method=HEH
=> homeo_method=HAP
In [83]:
print(homeo_methods, texts, colors)
for homeo_method, text, color in zip(homeo_methods, texts, colors):
    print(homeo_method, text, color)
['None', 'OLS', 'HEH', 'HAP'] ['No homeostasis', 'Olshausen (1997)', 'Histogram Equalization', 'Activation Probability'] ['black', 'orange', 'blue', 'red']
None No homeostasis black
OLS Olshausen (1997) orange
HEH Histogram Equalization blue
HAP Activation Probability red
In [84]:
texts = ["... find more info, and open-sourced code @ ",
    "https://laurentperrinet.github.io/publication/perrinet-19"]

txt_opts = dict(fontsize=50, font="Amiri-Bold", stroke_color='gray', stroke_width=.5, size=(W,H))
duration = 3
for text, color in zip(texts, colors):
    txt_clip = TextClip(text, color=color, **txt_opts)
    txt_clip = txt_clip.set_start(t).set_pos('center').set_duration(duration)
    t += duration
    clip.append(txt_clip)
      
In [85]:
clip
Out[85]:
[<moviepy.video.VideoClip.TextClip at 0x14348f310>,
 <moviepy.video.VideoClip.TextClip at 0x144882dd0>,
 <moviepy.video.VideoClip.TextClip at 0x14348f4d0>,
 <moviepy.video.VideoClip.ImageClip at 0x1434935d0>,
 <moviepy.video.VideoClip.ImageClip at 0x143409090>,
 <moviepy.video.VideoClip.ImageClip at 0x143409e10>,
 <moviepy.video.VideoClip.ImageClip at 0x143409fd0>,
 <moviepy.video.VideoClip.ImageClip at 0x143409250>,
 <moviepy.video.VideoClip.ImageClip at 0x143402510>,
 <moviepy.video.VideoClip.ImageClip at 0x143402f90>,
 <moviepy.video.VideoClip.ImageClip at 0x143402a10>,
 <moviepy.video.VideoClip.ImageClip at 0x143402fd0>,
 <moviepy.video.VideoClip.ImageClip at 0x143402b90>,
 <moviepy.video.VideoClip.ImageClip at 0x143402ad0>,
 <moviepy.video.VideoClip.ImageClip at 0x1433f9a50>,
 <moviepy.video.VideoClip.TextClip at 0x143417810>,
 <moviepy.video.VideoClip.ImageClip at 0x1433f9f90>,
 <moviepy.video.VideoClip.ImageClip at 0x1433f9c10>,
 <moviepy.video.VideoClip.ImageClip at 0x1433f9710>,
 <moviepy.video.VideoClip.ImageClip at 0x1433f92d0>,
 <moviepy.video.VideoClip.ImageClip at 0x14348f510>,
 <moviepy.video.VideoClip.ImageClip at 0x14348fa10>,
 <moviepy.video.VideoClip.ImageClip at 0x29d1a3290>,
 <moviepy.video.VideoClip.ImageClip at 0x1433f3990>,
 <moviepy.video.VideoClip.ImageClip at 0x1433f3290>,
 <moviepy.video.VideoClip.ImageClip at 0x29d1a06d0>,
 <moviepy.video.VideoClip.ImageClip at 0x144884e90>,
 <moviepy.video.VideoClip.ImageClip at 0x144884750>,
 <moviepy.video.VideoClip.TextClip at 0x1448841d0>,
 <moviepy.video.VideoClip.ImageClip at 0x14340e390>,
 <moviepy.video.VideoClip.ImageClip at 0x14341a7d0>,
 <moviepy.video.VideoClip.ImageClip at 0x14341aad0>,
 <moviepy.video.VideoClip.ImageClip at 0x14341a690>,
 <moviepy.video.VideoClip.ImageClip at 0x14341a610>,
 <moviepy.video.VideoClip.ImageClip at 0x14341a150>,
 <moviepy.video.VideoClip.ImageClip at 0x14342f310>,
 <moviepy.video.VideoClip.ImageClip at 0x14342f090>,
 <moviepy.video.VideoClip.ImageClip at 0x14342fa90>,
 <moviepy.video.VideoClip.ImageClip at 0x14342f9d0>,
 <moviepy.video.VideoClip.ImageClip at 0x14342f410>,
 <moviepy.video.VideoClip.ImageClip at 0x14342f510>,
 <moviepy.video.VideoClip.TextClip at 0x14342f6d0>,
 <moviepy.video.VideoClip.ImageClip at 0x143497510>,
 <moviepy.video.VideoClip.ImageClip at 0x143497d50>,
 <moviepy.video.VideoClip.ImageClip at 0x143497cd0>,
 <moviepy.video.VideoClip.ImageClip at 0x14342b250>,
 <moviepy.video.VideoClip.ImageClip at 0x14342bd10>,
 <moviepy.video.VideoClip.ImageClip at 0x14342b150>,
 <moviepy.video.VideoClip.ImageClip at 0x14342bf90>,
 <moviepy.video.VideoClip.ImageClip at 0x14342b450>,
 <moviepy.video.VideoClip.ImageClip at 0x14342b1d0>,
 <moviepy.video.VideoClip.ImageClip at 0x14342b950>,
 <moviepy.video.VideoClip.ImageClip at 0x1432fec90>,
 <moviepy.video.VideoClip.ImageClip at 0x1432fe1d0>,
 <moviepy.video.VideoClip.TextClip at 0x143422790>,
 <moviepy.video.VideoClip.TextClip at 0x143422150>]
In [86]:
      
# Overlay the text clip on the first video clip
video = CompositeVideoClip(clip)

# Write the result to a file (many options available !)
video.write_videofile(f'../files/{name}.{vext}', fps=5)
                                                              
t:   7%|▋         | 2/30 [30:08<7:01:52, 904.01s/it, now=None]
                                                              
t:   7%|▋         | 2/30 [30:08<7:01:52, 904.01s/it, now=None]
t:   7%|▋         | 2/30 [29:56<6:59:10, 898.24s/it, now=None]

t:   0%|          | 0/320 [00:00<?, ?it/s, now=None]

t:   1%|          | 3/320 [00:00<00:12, 26.18it/s, now=None]
Moviepy - Building video ../files/2019-09-11_Perrinet19.mp4.
Moviepy - Writing video ../files/2019-09-11_Perrinet19.mp4


t:   2%|▎         | 8/320 [00:00<00:10, 29.82it/s, now=None]

t:   4%|▍         | 13/320 [00:00<00:09, 33.64it/s, now=None]

t:   6%|▌         | 18/320 [00:00<00:08, 37.00it/s, now=None]

t:   7%|▋         | 23/320 [00:00<00:07, 39.88it/s, now=None]

t:   9%|▉         | 28/320 [00:00<00:06, 42.29it/s, now=None]

t:  10%|█         | 33/320 [00:00<00:06, 44.04it/s, now=None]

t:  12%|█▏        | 38/320 [00:00<00:06, 44.84it/s, now=None]

t:  13%|█▎        | 43/320 [00:00<00:06, 45.46it/s, now=None]

t:  15%|█▌        | 48/320 [00:01<00:05, 45.49it/s, now=None]

t:  17%|█▋        | 53/320 [00:01<00:05, 45.86it/s, now=None]

t:  18%|█▊        | 58/320 [00:01<00:05, 45.64it/s, now=None]

t:  20%|█▉        | 63/320 [00:01<00:05, 45.73it/s, now=None]

t:  21%|██▏       | 68/320 [00:01<00:05, 45.97it/s, now=None]

t:  23%|██▎       | 73/320 [00:01<00:05, 46.15it/s, now=None]

t:  24%|██▍       | 78/320 [00:01<00:05, 45.73it/s, now=None]

t:  26%|██▌       | 83/320 [00:01<00:05, 45.58it/s, now=None]

t:  28%|██▊       | 88/320 [00:01<00:05, 45.41it/s, now=None]

t:  29%|██▉       | 93/320 [00:02<00:04, 45.56it/s, now=None]

t:  31%|███       | 98/320 [00:02<00:04, 45.48it/s, now=None]

t:  32%|███▏      | 103/320 [00:02<00:04, 45.66it/s, now=None]

t:  34%|███▍      | 108/320 [00:02<00:04, 45.95it/s, now=None]

t:  35%|███▌      | 113/320 [00:02<00:04, 45.64it/s, now=None]

t:  37%|███▋      | 118/320 [00:02<00:04, 45.76it/s, now=None]

t:  38%|███▊      | 123/320 [00:02<00:04, 45.63it/s, now=None]

t:  40%|████      | 128/320 [00:02<00:04, 45.61it/s, now=None]

t:  42%|████▏     | 133/320 [00:02<00:04, 45.80it/s, now=None]

t:  43%|████▎     | 138/320 [00:03<00:03, 45.75it/s, now=None]

t:  45%|████▍     | 143/320 [00:03<00:03, 45.90it/s, now=None]

t:  46%|████▋     | 148/320 [00:03<00:03, 45.95it/s, now=None]

t:  48%|████▊     | 153/320 [00:03<00:03, 46.03it/s, now=None]

t:  49%|████▉     | 158/320 [00:03<00:03, 46.18it/s, now=None]

t:  51%|█████     | 163/320 [00:03<00:03, 46.23it/s, now=None]

t:  52%|█████▎    | 168/320 [00:03<00:03, 46.38it/s, now=None]

t:  54%|█████▍    | 173/320 [00:03<00:03, 46.05it/s, now=None]

t:  56%|█████▌    | 178/320 [00:03<00:03, 45.72it/s, now=None]

t:  57%|█████▋    | 183/320 [00:04<00:02, 46.01it/s, now=None]

t:  59%|█████▉    | 188/320 [00:04<00:02, 45.68it/s, now=None]

t:  60%|██████    | 193/320 [00:04<00:02, 45.72it/s, now=None]

t:  62%|██████▏   | 198/320 [00:04<00:02, 45.86it/s, now=None]

t:  63%|██████▎   | 203/320 [00:04<00:02, 45.63it/s, now=None]

t:  65%|██████▌   | 208/320 [00:04<00:02, 45.39it/s, now=None]

t:  67%|██████▋   | 213/320 [00:04<00:02, 45.21it/s, now=None]

t:  68%|██████▊   | 218/320 [00:04<00:02, 45.49it/s, now=None]

t:  70%|██████▉   | 223/320 [00:04<00:02, 45.66it/s, now=None]

t:  71%|███████▏  | 228/320 [00:04<00:02, 45.85it/s, now=None]

t:  73%|███████▎  | 233/320 [00:05<00:01, 45.95it/s, now=None]

t:  74%|███████▍  | 238/320 [00:05<00:01, 46.03it/s, now=None]

t:  76%|███████▌  | 243/320 [00:05<00:01, 45.94it/s, now=None]

t:  78%|███████▊  | 248/320 [00:05<00:01, 45.94it/s, now=None]

t:  79%|███████▉  | 253/320 [00:05<00:01, 46.03it/s, now=None]

t:  81%|████████  | 258/320 [00:05<00:01, 45.75it/s, now=None]

t:  82%|████████▏ | 263/320 [00:05<00:01, 45.53it/s, now=None]

t:  84%|████████▍ | 268/320 [00:05<00:01, 44.33it/s, now=None]

t:  85%|████████▌ | 273/320 [00:05<00:01, 44.07it/s, now=None]

t:  87%|████████▋ | 278/320 [00:06<00:00, 43.51it/s, now=None]

t:  88%|████████▊ | 283/320 [00:06<00:00, 43.87it/s, now=None]

t:  90%|█████████ | 288/320 [00:06<00:00, 43.76it/s, now=None]

t:  92%|█████████▏| 293/320 [00:06<00:00, 43.32it/s, now=None]

t:  93%|█████████▎| 298/320 [00:06<00:00, 43.42it/s, now=None]

t:  95%|█████████▍| 303/320 [00:06<00:00, 44.23it/s, now=None]

t:  96%|█████████▋| 308/320 [00:06<00:00, 44.58it/s, now=None]

t:  98%|█████████▊| 313/320 [00:06<00:00, 44.64it/s, now=None]

t:  99%|█████████▉| 318/320 [00:07<00:00, 45.37it/s, now=None]

                                                              
t:   7%|▋         | 2/30 [30:15<7:03:33, 907.61s/it, now=None]
                                                              
t:   7%|▋         | 2/30 [30:15<7:03:33, 907.61s/it, now=None]
t:   7%|▋         | 2/30 [30:03<7:00:51, 901.84s/it, now=None]
Moviepy - Done !
Moviepy - video ready ../files/2019-09-11_Perrinet19.mp4
In [37]:
%pwd
Out[37]:
'/Users/laurentperrinet/pool/blog/laurentperrinet.github.io_sciblog/posts'

some book keeping for the notebook



In [ ]:
%load_ext version_information
%version_information numpy, scipy, matplotlib, MotionClouds

Comments

Comments powered by Disqus